1. Field of the Invention
The present invention relates to a neural network in which, when learning is performed, the value of a synaptic weight can be externally and forcedly re-set.
2. Description of the Prior Art
In recent years, a multi-layer neural network in which the learning is performed by the error back-propagation learning method has been utilized in a field of speech or character recognition. A neural network usually used in such a field is of a multilayer perceptron type, typically, the three-layer perceptron type. As shown in FIG. 2, a three-layer perceptron type neural network comprises an input layer 21, an intermediate layer 22, and an output layer 23.
Units 25 in the input layer 21, units 26 in the intermediate layer 22, and units 27 in the output layer 23 are synaptically connected so as to constitute a network. When data are input into the units 25 of the input layer 21, data in accordance with the configuration of the network are output from the units 27 of the output layer 23. Each of these units comprises a portion for receiving data input from another unit, a portion for converting the input data based on a predetermined rule, and a portion for outputting the converted result to a unit which constitutes the upper layer. To a synaptic connection between the units, a weight indicative of the connecting strength is applied. Hereinafter, such a connection weight is referred to as "a synaptic weight". If the value of the synaptic weight is changed, the configuration of the network changes so that the network outputs a different output value for the same input data.
The learning of the neural network is carried out as follows. Learning data which belong to a known category are input into the units 25 of the input layer 21, and teaching data or target indicative of the category to which the input data belong are supplied to the units 27 of the output layer 23. The synaptic weights are set so that the units 27 of the output layer 23 output the same data as the teaching data. Generally, such learning is performed by the error back-propagation learning method.
When data representative of the feature pattern of an object to be recognized are input into the units 25 of the input layer 21, the thus learned neural network outputs data representative of a category to which the input data belong, from the units 27 of the output layer 23. Accordingly, based on the category represented by the output data, the category to which the input data (i.e., the feature pattern) belong can be accurately identified.
Since the number of the units 26 of the intermediate layer 22 in the above-mentioned three-layer perceptron type neural network is determined by trial and error, the number of units of the intermediate layer 22 is generally set to be substantially larger than a number which may be necessary for achieving a required recognition accuracy. As a result, the calculation time period for the entire neural network is longer, or some of the units do not affect the learning or recognition. This causes a problem in that the efficiency in the learning or recognition is decreased.
The inventor and Mr. Togawa have proposed an apparatus for controlling the learning of a neural network (Japanese Patent Application No. 1-52684 which is now published as Japanese Patent Publication (Kokai) No. 2-231670 and U.S. patent application Ser. No. 07/486,165 filed on Feb. 28, 1990) now U.S. Pat. No. 5,195,169, which is shown in FIG. 4. This proposed apparatus comprises a monitor 4 which monitors the progress of the learning in the neural network, as shown in FIG. 4. In FIG. 4, units in the respective layers, connections between respective units, synaptic weights Wd and Wu, input data, and output data of the three-layer perceptron type neural network of FIG. 2 are shown in a simplified manner.
In this system, a synaptic weight Wd is applied to the synaptic connection of each of the units in the intermediate layer 22 which are connected with all of the units in the input layer 21, and a synaptic weight Wu is applied to the synaptic connection of each of the units in the output layer 23 which are connected with all of the units in the intermediate layer 22. The monitor 4 receives: the values of the synaptic weights Wd and Wu; output data from each of the units in the output layer 23; and teaching data, and monitors the values of the synaptic weights Wd and Wu in the learning process of the neural network. For example, in the cases where units not affecting the learning exist when the learning is nearly converged, where units exhibiting a similar synaptic weight exist when the learning is nearly converged, or where units exhibiting a similar synaptic weight when the learning is not converged, the values of the synaptic weights Wd and Wu corresponding to these units are set to zero. Thus, the learning efficiency can be increased, and the number of units in the intermediate layer 22 can be modified to the optimum number, so that the optimum configuration of the neural network is established.
In short, in the above-mentioned apparatus, the values of the synaptic weights Wu and Wd are monitored, and the values of the synaptic weights Wu and Wd are set to zero. In such an apparatus, however, the values of the synaptic weights Wu and wd which can be monitored by the monitor 4 are limited to those of the synaptic weights Wu and Wd at a certain learning point. Accordingly, the existence of units which do not affect the learning or which exhibit a similar synaptic weight value is judged based on the values of the synaptic weights Wu and Wd at the present learning point.
Practically, the progress of the learning is indicated by the combination of the values of the synaptic weights Wu and Wd at an interested learning point and the past record of the values of the synaptic weights Wu and Wd. Accordingly, there arises a problem that the progress of the learning cannot be appropriately judged by the values of the synaptic weights Wu and Wd at a certain learning point.